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[57] ABSTRACT 

A system and method for providing on-line, real-time, 
transparent data migration from an existing storage device to 
a replacement storage device. The existing and replacement 
storage devices are connected as a composite storage device 
that is coupled to a host, network or other data processing 
system. The replacement storage device includes a table 
which identifies data elements that have migrated to the 
replacement storage device. When a host system makes a 
data transfer request for one or more data elements, the 
replacement storage device determines whether the data 
elements have been migrated. If the data elements have 
migrated, the replacement storage device responds to the 
data transfer request independently of any interaction with 
the existing storage device. If the data elements have not 
migrated, the replacement storage device migrates the 
requested data elements and then responds to the data 
request and updates the data element map or table. When not 
busy servicing other requests, the replacement storage 
device operates in a background mode to migrate data 
elements so the data migration can occur concurrently with 
and transparently to system operations. 
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SYSTEM AND METHOD FOR ON-LINE, 
REAL TIME, DATA MIGRATION 

CROSS-REFERENCE TO RELATED 

APPUCAnON 5 

This is a continuation-in-part of application for U.S. 
patent Ser. No. 08/522,903 filed Sep. 1, 1995, now U.S. Pat. 
No. 5,680.640 issued Oct. 21, 1997, for a System and 
Method for On-Line, Real Time, Data Migration. 

BACKGROUND OF THE INVENTION 

1. Field of the Invention 

This invention relates to data storage systems and more 
particularly, to a system and method for on-line replacement 15 
of an existing data storage subsystem. 

2. Description of Related Art 

Data processing centers of businesses and organizations 
such as banks, airlines and insurance companies, for 
example, rely almost exclusively on their ability to access 
and process large amounts of data stored on a data storage 
device. Data and other information which is typically stored 
on one or more data storage devices which form part of a 
larger data storage system is commonly referred to as a 
database. 

Databases are nearly always "open" and constandy "in 
use" and being accessed by a coupled data processing 
system, central processing unit (CPIJ) or host mainframe 
computer. The inability to access data is disastrous if not a 
crisis for such business and organizations and will typically 
result in the business or organization being forced to tem- 
porarily cease operation. 

During the course of normal operations, these businesses 
and organizations must upgrade their data storage devices 35 
and data storage systems. Although such upgrading some- 
times includes only the addition of data storage capacity to 
their existing physical systems, more often than not upgrad- 
ing requires the addition of a completely separate and new 
data storage system. In such cases, the existing data on the ^ 
existing data storage system or device must be backed up on 
a separate device such as a tape drive, the new system 
installed and connected to the data processing unit, and the 
data copied from the back-up device to the new data storage 
system. Such activity typically takes at least two days to 45 
accomplish. If the conversion takes more than two days or 
if the business or organization cannot withstand two days of 
inoperability, the need and desire to upgrade their data 
storage system may oppose an insurmountable problem. 

Some prior art data copying methods and systems have 50 
proposed allowing two data storage systems of the same 
type, a first system and a second system, to be coupled to one 
another, and allowing the data storage systems themselves to 
control data copying from the first to the second system 
without intervention from or interference with the host data 55 
processing system. See for example, the data storage system 
described in U.S. Pat. No. 5,544,347 entitled REMOTE 
DATA MIRRORING, fully incorporated herein by 
reference, which describes one such remote data copying 
facility feature which can be implemented on a Symmetrix 50 
5500 data storage system available from EMC Corporation, 
Hopkinton, Mass. 

Although such a system and method for data copying is 
possible, in most instances, the first and second data storage 
systems are not of the same type, or of a type which allow 65 
such a "background" data migration to take place between 
the two data storage systems, imassisted by the host and 
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while the database is open. Additionally, even on such prior 
art data storage systems, migrating data as a "background" 
task while the database is "open" does not take into account 
the fact that the data is constantly changing as it is accessed 
by the host or central processing unit and accordingly, if the 
old system is left connected to the host, there will always be 
a disparity between the data which is stored on the old data 
storage system and the data which has been migrated onto 
the new data storage system. In such cases, the new data 
storage system may never fiilly "catch up" and be able to be 
completely synchronized to the old data storage system. 

Accordingly, what is needed is a system and method for 
allowing data migration between a first data storage system 
and a second data storage system while the database is open 
and a real-time data migration that is completely transparent 
to the host or data processing unit. 

SUMMARY 

This invention features a system and method for provid- 
ing on-line, real-time, transparent data migration between 
two data storage devices. The system includes a first data 
storage device which was previously coupled to an external 
source of data including a data processing device such as a 
host computer, or a network which may be connected to a 
number of data processing devices such as a number of host 
computers. The data processing device such as a host 
computer reads data from and writes data to the data storage 
device. The first data storage device initially includes a 
plurality of data elements currently being accessed by the 
data processing device. 

At least one second data storage device is provided which 
is coupled to the first data storage device and to the data 
processing device, for storing data elements to be accessed 
by the data processing device. The second data storage 
device preferably includes a data element map including at 
least an indication of whether or not a particular data 
element is stored on the second data storage system. 

In one embodiment, the second data storage system 
independently migrates data from the first to the second data 
storage system independent of the source. In another 
embodiment, the second data storage system is responsive to 
the external source, for migrating data from the first to the 
second data storage system. 

In yet another embodiment, the data processing device 
issues a data read request (in the case of a read data 
operation), or a data write command (in the case of a write 
operation). The request is received by the second data 
storage device. In the case of a read operation, second data 
storage device examines the data map or table to determine 
whether or not the data has been migrated to and is stored on 
the second data storage device. If it is determined that the 
data is stored on the second data storage device, the data is 
made available to the requesting device. 

If the data is not stored on the second data storage device, 
the second data storage device issues a data request, in the 
form of a read data command, to the first data storage device, 
obtains the data and makes the data available to the request- 
ing device. The data received from the first data storage 
device is also written to the second data storage device and 
the data map updated. 

In the case of a write operation, one embodiment con- 
templates that if the data received from the data processing 
device is destined for a location on the data storage system 
that has not yet been copied or 'migrated* from the older or 
first data storage device (a data storage location marked in 
the data map as *need to migrate'), and the data is not a full 
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or complete data element (for example, not a 'full track' of been migrated to the data store by using a second, higher 

data) the write operation is suspended, the "complete" data priority mode, to transfer the requested data from the donor 

element from the corresponding location (a *fuli track"' for disk array storage system and to provide the requested data 

example) on the first data storage device is read into the to the host computer 

cache memory on the se«,nd data storage device, the 5 ^^^^^ , ^^other aspect of this invention 

m-cache flag or bit set, the data storage location marked or •.j.r. j ^-i . 

identified ^ 'write pending', and the write operation data migrates data from a donor disk array storage system to 

resumed meaning that the data wiU be 'written' to and over » rfPla^ement disk array storage system. The replacement 

the 'full track' of data now stored in the cache memory of the »"»y s'^f^g* sy««°' continues to respond to write 

second data storage system. In other embodiments, the older ,„ f*?"**!* computer during the migraUon. The 

data may not be retrieved from the first or older data migration includes comiecting the replacement disk array 

processing device if the new data to be written is known to *'°"8e system to the host computer and to the donor disk 

be a complete data element (a 'full track' for example). *'°f*8« *y^'«™ '° mput-output requests from 

When the second data storage device is not busy handling ^•'"f J. ''"""^ '^^^ "^"y 

data read or write requests from a coupled data processing system ceases to respond direcdy to input-output requests 

device, such as a host computer, the second data storage ^""^ '^^ "'T ' t "'^f'"' ^"^^^t 

system examines its data map/table to determine which dala f * '^'"y ^yf" '° replacement disk 

elements are resident on the Lt data storage device and are V'^J "y"'"'" ""^ repla^ment 

not stored on the second daU storage deWce. The second f "7 storage system concuirenUy with and indepen- 

data storage device then issues read requests to the first data ,„ ^ent^y of any mput-output requests directed to the rcplace- 

storage device requesting one or more of those data '° nient disk array storage system. Write request from the host 

elements, receives the data, writes die daU to the second data '=°'°P"'" ^^V'T, '° '^"^ ^P'^^ment disk array storage 

storage device and updates the data map/table to indicate fy*'%°» '.° ^^«=^^ "l?' ^^'"^^ 

that the data is now stored on the second data storage device. tr^nsferTrng the data associated with the write request to the 

J *u • J* c replacement disk array storage system and by preventing the 

In this manner, there IS no need to perform lime consum- 9<: * r r j- c .i_ j j- i 

„ . * . . , a . J J J . transfer of corresponding data from the donor disk array 

mg oli-une data migration between first and second data * * . *u i * j • i * * 

, , • u « *i. *u J * • • vuv* viaia storage system to the replacement disk array storage system, 

storage devices but rather, the data copying or migration can j & j 

occur in real-time, while the data storage devices are on-line ^" accordance with still yet another aspect of this 

and available to the host or other requesting device, and "i^enuon, apparatus migrates data from a donor disk array 

completely transparent to the coupled data processing 30 ^^^^^^ ^° ^ replacement disk array storage system 

^Qyice while continuing to respond to write requests from a host 

In accordance with this invention, data is migrated from computer coupled to the replacement disk array storage 

a first disk array storage system that responds to input-output ^y^^^"^: "PP"^^"^ ^ ^ ^^^^ 

requests including read requests from a host computer to a migration control connects to the host computer 

second disk array storage system that is to replace the first 35 ^"""^ ^^""l^f ^^^^^"^ ^^^'^^^ '"'^''^^^ 

disk array storage system while continuing to respond to "^P^t=output requests from the host computer to the replace- 

read requests from the host computer. The steps of this difk array storage system. Data migrates data from the 

method include redirecdy inputK)utput requests from the "^'^ ^^^^ system to the replacememt disk 

host computer to the second disk array storage systen ^"f ^ "'^'^^e system under the control of the replacement 

whereby the first disk array storage system ceases to respond 40 ^"^^ ^'^^ '^'''^^^ T'"""; replacement disk array slor- 

directly to input-output requests from the host computer. system responds to data write requests from the host 

DaU migrates from the first disk array storage system to the ^^"'P^^^' replacement disk array storage 

second disk array storage system under the control of the P'."^ ^ ^^^^ 7^^ "^'^^^^ 

second disk array storage system concurrently with and the data associated with the write request to the 

independently of any input-output requests directed to the 45 [^P^^f "^^^ ^"^^ array storate system and by preventing the 

second disk array storage system by the host computer. The corresponding data from the donor disk array 

second disk array responds to a data read request from the "^^'^^e system to the replacement disk array storage system, 

host computer for data that has not yet been migrated to the BRIEF DESCRIPTION OF THE DRAWINGS 
second disk array storage system by transferring the 

requested data from the first disk array storage system 50 appended claims particularly point out and distinctly 

through the second disk array storage system to the host subject matter of this invention. The various 

computer whereby after the migration is complete the sec- objects, advantages and novel features of this invention will 

ond disk array storage system replaces the first disk array ^® apparent from a reading of the foUowing 

storage system. detailed description in conjunction with the accompanying 

In accordance with another aspect of this invention, a 55 drawings in which like reference numerals refer to like parts, 

replacement disk array storage system for connection to a ^° which: 

host computer replaces a donor disk array storage system ^IG. 1 is a schematic diagram of an exemplary data 

containing data. The replacement disk array storage system processing and data storage system on which the system and 

comprises a data store, and a migration control that connects method for providing on-line, data transparent data migra- 

to the host computer and to the donor disk array storage 60 between first and second data storage systems in accor- 

system thereby redirecting input-output requests to the ^^^mce with the present invention may be accomplished; 

replacement disk array storage system and that operates in a FIG. 2 is a schematic illustration of a data element map or 

first mode by migrating data from the donor storage system table; 

to the data store concurrently with and independently of any FIG. 3 is a flowchart outlining the steps of providing 

input-output requests from the host computer. The migration 65 on-line, transparent data migration between first and second 

control responds to a read request from the host computer data storage systems according to the method of the present 

during the dara migration if the requested data has not yet invention; 
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FIG. 4 is a flowchart illustrating the steps for providing second data storage system 16. The data map or table is 

data migration between first and second data storage systems established during the set-up or configuration of the second 

without data storage device or host system intervention data storage system 16 and is dependent on the particular 

when the second data storage device is not busy handUng configuration of the second data storage system 16. 
data requests from the host or data processing device; 5 Preferably, the data map/table 24 also includes informa- 

FIG. 5 is a schematic diagram of another embodiment of tion about data elements which are stored in the first data 

a data processing and data storage system on which the storage system 14, the use of such a data map/table will be 

system and method for providing on-line, data transparent explained in greater detail below. 

data migration between first and second data storage systems The second data storage system 16 is typically and 
in accordance with the present invention may be accom- 10 preferably coupled to a data storage system configuration 

pUshed; device 26 such as a computer, which allows the user to 

FIG. 6 is a flowchart illustrating the steps for connecting configure the second data storage system 16 and the data 

the second data storage system without interrupting the map/table 24 as desired by the user. In the preferred 

operation of the data processing system; embodiment, the second data storage system 16 is at least 

HG. 7 is a detailed flowchart illustrating the steps of a partially configured exactly as the first data storage system 

procedure of FIG 6* configured in terms of the number of logical devices, 

RG. 8 is a schematic diagram of another embodiment of J°"Se size, storage system type (3380/3390, for example) 
a data processing and data storage system incorporating this 

invention* preferred embodunent, the data storage system 

rt - a L ^ -It . . r 1 J • configuration device 26 allows the user to configure at least 

FIG. 9 IS a flowchart illustratmg the steps for shadowmg t. ^ *t, j * * j j * * 

f - • T-ryT o ^ a portion or the data storage area on second data storage 

the operation of the circuit m FIG. 8; « i r . • i ^ ^ * i . . i 

^ ' system 16 to include data element storage locations or 

HG. 10 depicts a set of registers that are useful in addresses which correspond to data element storage 

accordance with another aspect of this invention; addresses on the first data storage system 14. 

HG. 11 is a flow chart of the steps for a copy block in the preferred embodiment, the second data storage 

program operating in accordance with this other aspect of system 16 is a disk drive data storage system employing a 

this invention; large number of fixed block architecture (FBA) formatted 

FIG. 12 is a flow chart of a program that controls an disk drives 10, data stores, 17«-17«, and adapted for storing 

operating mode for the copy block program of FIG. 11; and large amounts of data to be accessed by a host computer or 

FIG. 13 graphicaUy depicts the advantages of the imple- other data processing device 12. The exemplary second data 

mentation of FIGS. 10 through 12. storage system 16 also typically includes a cache memory 18 

which serves to hold or buffer data read and write requests 

DESCRIPTION OF ILLUSTRATIVE between the second data storage system 16 and the host or 
EMBODIMENTS 35 other data processing device 12. Such data storage systems 

_ . . „ are weU known to those skilled in the art and include, for 

Hie present mvention features a system and method for example, the Symmetrix 5500 series data storage system 

providing on-lme. real-tune, transparent data migration available from EMC Corporation, Hopkinton, Mass., a 

between two data storage systems, at least one of which is description of which is incorporated herein by reference, 

coupled to a data processing device such as a host computer. t n j . . • ^ . 

40 Initially, the second or new data storage system 16 is first 

An exemplary system 10, FIG. 1 on which the present coupled to the first data storage system 14 by means of one 
mvention may be perforated and implemented includes a or more data communication links or paths 20a, 2Qb. After 
host computer, central processing unit or other simflar data the second data storage system 16 has been configured using 
processing device 12. The data processmg device 12 is a system configuration device 26 or other similar or equiva- 
mitially coupled to a first data storage system 14. In most lent device, or by the host 12, the second data storage system 
mstances, the first data storage system 14 is an older data coupled to the host computer 12 or other data process- 
storage system which is either not large enough to handle the ing device by means of a data communication path 28. , 
needs of the data processing device 12, or for some other preferably, data communication path 28 is a high speed I"' 
reason ^ going to be comp etely or partially replaced or communication path such as a fiber optic "ESCON" com- 1 
augmented by the addition of a second data storage system ... ♦u uu u j 11 4U • II 
^ ^ ^ 50 mumcation path, although any and all other commumcation jj 

paths are considered to be within the scope of the present U 

The first data storage system 14 is initially coupled to the invention. Immediately before connecting data communica- \ 

data processing device 12 by means of a data communica- tion path 28 between the host or other data processing unit 

tion link 19. The second data storage system 16 is coupled 12 in the second data storage system 16, the previously 
to the first data storage system 14 by means of one or more 55 existing data communication path 19 between the host 12 

data commumcation paths 20a, and/or 20b. Examples of and the first data storage system 14 is disconnected or 

data communication paths 20fl-20^ include an IBM "bus severed as iUustrated at arrow 30, 

and tag" connection well known to those skilled in the art, j^^^ ^^^^^ ^^^^ jhe prior art whereby the host or 

^^ndhigherspeed fiber oph c conne cti ons such as anjSCO N ^ther data processing system 12 must be taken off line for a 
- ~ ■ 60 number of days in order to allow for backing up of data on 

If the first and second data storage systems 14, 16 have an the first data storage system 14 followed by the replacement 

incompatible data communication protocol or interface, a of the first data storage system 14 with a second data storage 

protocol converter 22 may be provided on one or more of the system 16 and subsequent copying of all of the data onto the 

data communication links 20a, 20b as required, and as is new data storage system 16, or a host which remains coupled 
well known in the art. ^5 jq the original 'first' data storage system 14, the present 

The second data storage system 16 includes a data map or invention only requires the host computer or other data 

table 24 of data elements which are stored on at least the processing device 12 to be off line or service interrupted for 
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a relatively short period of time (the procedure typically "migrates" the data from the first data storage system 14 to 

takes approximately 10 minutes or less), while the first data the second data storage system 16 completely transparent to 

signal path 19 is severed or disconnected and the second data the host 12, and often in paraliei with the channel process 25 

signal path 28 is established between the second or new data which may be retrieving data from the first data storage 

storage system 16 and the host computer or other data 5 system 14 in response to requests from the host or CPU 12, 

processing device 12. while maintaining fiill accessibility to the data by the host or 

Accordingly, after the second data storage system 16 has ""^^^^ ^^^^ processing device 12. 

been connected to the host or other data processing unit 12, ^ exemplary data element map/table 24 is shown m 

wheneverthehostordataprocessingunitl2issuesarequest f"^^^' ^' ^.^ preferred embodiment, the 

to read data from or write data to "its" data storage system, lo ^ata map/table 24 is organized in a hierarchical fashion. For 

■ ju *u J J . * / ^/ example, for the preferred embodiment wherem the data 

requ.st is reoeiv^ by the second data storage system 16. ^ j^^,^^^ ^ j^^j ^ 

Usmg a bit or flag from Oie data map/table 24 previously devices such as disk drives lla-lln, and wherein 

estabhshed and configured, the second data storage system ^^^^^ j^ive is partitioned into one or more logical 

16. by scannmg data mapNtable 24, determines whether or ..volumes" and each volume comprises a number of disk 

not the data requested (in the case of a read operation) is « ^^^^ ^^^^^^ ^^^^ mapAable 24 wiU first have an entry 50 

stored on the lirst data storage system 14 or on the second for each physical and/or logical device such as a disk drive, 

ata storage system . , , , ^ ^ . ^ , , . , The device entry 50 will be followed by an entry 52 for 

Such a hierarchical data map/table 24 is further explamed ^ logical volume, followed by one or more entries 

and exemplified herein as well as in U.S. Pat. Nos. 5,206,939 54^.54^ fo^ each track of the device which comprises the 

and 5,381,539 assigned to the assignee of the present 20 ^^^.^^ ^^^^^^ ^^^^^ 54a-54c for the first 

mvention and both fuUy incorporated herein by reference. ^^^^^ ^ ^ followed by entry line 56 for the second 

If the data is ah-eady stored in the second data storage logical volume configured on the physical device indicated 

system 16, the second data storage 16 retrieves the data by the entry at line 50. 

(perhaps temporarily storing the data in cache memory 18) All information about the data storage system and each 

as IS well known in the art, and makes the data available to device in the data storage system with the exception of the 

the host or other requesting data processing device 12. «dala in cache" indication flag or bit 58 is stored in hierar- 

If the requested data is not on the second data storage chical format in the data map/table 24. Thus, whenever the 

system 16, channel or real-time data handling process 25 of second data storage system 16 desires or needs to obtain 

the second data storage system 16 issues a read data request information about a particular data element (be it an indi- 

to the first data storage system 14 in the manner and format vidual data record, track or volume), the data storage system 

native or known to the first data storage system 14 (for 16 scans the data map/table 24 beginning at the device level 

example, sUndard IBM data read commands). Channel or 50 to determine whether or not the desired criterion or 

real-time data handling process 25 is, in the preferred characteristic has been established for any track or volume 

embodiment, a software program comprising a series of of a device. 

commands or instructions which receives one or more xhere wiU be a *flag' or other similar indicator bit set, or 

commands from the second data storage system interface to other indication of the desired characteristic in the device 

the host or CPU (typicaUy called a "channel"), mterprets entry 50, in the volume entry 52 and in the appropriate track 

those commands, and issues one or more corresponding entry 54 if the desired characteristic is found in that portion 

commands which can be acted upon by the first data storage ^ of the data storage device represented by the data map/table 

system. Such an * interpreter' type of software is well known 24. 

to those skilled in the art. example, the preferred embodiment of a data map/ 

The first data storage system 14 then retrieves the table 24 includes a write pending flag or bit 61 which is set 

requested data and provides it to the second data storage if a particular data element is presently stored in cache 18 of 

system 16. The second data storage system 16 then makes 45 the second data storage system 16 and must be written to 

the data available to the host or other data processing unit 12 bnger term storage such as a disk drive \la-\ln. For 

which has requested the data. exemplary purposes, assuming that track 2 of volume 1 is in 

Since the second data storage system now has a copy of cache 18 in the second data storage system 16 and write 

the data, the data will be written to the second data storage pending, the write pending flag or bit 61 and the in cache bit 

system 16 and the appropriate data map/table 24 flags or bits 50 58 at line entry 546 (for track two) will be set, as wiU the 

updated to indicate that the data has been migrated to the write pending bit 61 of volume 1 at line 52 of the data 

second data storage system 16 so that next time the same map/table 24, as will the write pending bit 61 of the device 

data element is requested, the second data storage system 16 at line 50. 

will have the data already stored on the system and will not Thus, if the second data storage system 16 wishes to 

have to request it from the first data storage system. 55 determine whether or not a particular track or record which 

Further, as will be explained in greater detail below, the has been requested is write-pending or has been migrated to 

second data storage system 16 can perform a "background" the second system or of the status of some other attribute or 

data migration procedure or process 27. The "backgroimd" characteristic, the data storage system 16 first determines 

data migration procedure of process 27 is, in the preferred which device or disk drive lla-Vln the data element is 

embodiment, a software program including a series on 60 stored on and then checks the appropriate indicator flag bit 

instructions which coordinate, monitor and control data for that device. If the particular indicator flag bit is not set 

migration whereby whenever the second data storage system for that device, then the second data storage system 16 

is not busy handling data input/output requests from the host knows immediately that no lower level storage unit or 

or other data processing device 12, the migrate process 27 of location such as a volume or track in that device has that 

the second data storage system 16 determines which data on 65 attribute. If any lower data storage element in the hierarchi- 

the first data storage system has not been copied by reading cal structure such as a track or volume includes the attribute, 

a specified flag or bit in its data table/map 24, and copies or than the attribute or flag bit for the device will be set. 
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Similarly, if a particular data storage location such as a Subsequently, the second data storage system 16 receives 

record or track which is part of a logical volume has the the requested data from the first data storage system 14, step 

requested attribute, then the corresponding attribute or flag 108 and writes the data to the cache memory IS of the 

bit for the volume will be set. The data storage system 16 can second data storage system 16 while updating the data 

thereby quickly detennine whether any data storage location 5 element map/table 24, step 110. The second data storage 

having a lower level than the volume or other similar logical system 16 then provides an indication to the host or data 

or physical partition being examined has the particular processing device 12 that the data is ready to be read, step 

attribute, without scanning or searching each and every 112. Subsequendy, the second data storage system 16 will 

lower level data storage location. write the data from cache memory 18 to a more permanent 

The "in-cache" flag or bit is an exception to the hicrar- lO storage location, such as a disk drive, on the second data 

chical structure in that since each line or entry 50-56 of the storage system 16, step 114, followed by a final update to 

data map/table 24 is directiy addressable, the second data one or more bits or flags of the data element map/table 24, 

storage system directly addresses the table entry line for a step 116. 

particular data element when it must inquire or "look-up" Thus, in the case where requested data is not yet stored on 

whether that particular data element is presently "in-cache". 15 the second data storage system 16, the "read request" 

It is understood, however, that this flag or bit could be command from the host 12 results in the second data storage 

managed in a hierarchical fashion without departing from system 16 "migrating' the data from the first data storage 

the scope of this invention. system 14 to the second data storage system 16. 

In addition to the in-cache bit or flag 58 and the write If the host or other data processing system 12 issues a 

pending flag or bit 61, the data map/table 24 which is one ^0 write request or command, step 120, the channel process 25 

feature of die present invention includes, in the preferred of the second data storage system 16 determines if the data 

embodiment, other flag bits 62 such as an invahd track to be written has been previously migrated from the first to 

format flag or bit, and an indication of whether or not data the second data storage system, step 122. If the data has been 

on a particular device, volume or track needs migration or previously migrated, step 122, the second data storage 

has been migrated from the first to the second data storage ^5 system writes the data to cache and updates any necessary 

system 14yl6 respectively, as shown generaUy by flag or flags or bits in the data map/table 24, step 110. Processing 

indicator bit 60. continues as previously described. 

Data map/table 24 may further include a physical address if, however, the data has not been previously migrated, 

64 entry for each element in the map or table 24, which step 122, the method of the present invention next 

identifies the beginning data address 64 at which the corre- determines, by the type of command or request issued by the 

sponding data element can be found on the disk drive host (for example in the case of IBM host commands), 

17a-17n of the new or second data storage system 16. whether or not the write request is for a fuU or complete data 

The operation of the method according to the present element storage location, such as a full or complete "track" 

invention wiU be described in greater detail beginning with of data, step 124. If the write request is for a fiiU "track" or 

step 100, FIG. 3, wherein the second data storage system 16 other similar type of data block or content, the second data 

receives a data element read or write request from the host storage system does not need to worry about migrating the 

or other data processing device 12, step 100. The method data from the first data storage system 14 since all the "old" 

next determines if the request or command is a read or a data is being replaced by the current command and 

write request, step 101. If the command is a read command, ^ therefore, processing continues to step 110 as previously 

the channel handling process 25 of the second data storage described. 

system 16 next determines if the requested data is ahready If however, the method determines that the write request 

stored in the second data storage system 16, step 102, by is for less then a full or complete data block or confine, such 

reading its data table map/table 24. as a track, step 124, the method next temporarily suspends 

If the data is stored on the second data storage system, 45 handling of the write request, step 126 and issues a "read" 

step 102, the second data storage system 16 will make the conunand for the full or complete "track" to the first data 

data available to the host or other requesting data processing storage system 14, and reads a predetermined amount of data 

device 12, step 104, and return to step 100 to await receipt (a whole track of data for example), step 128, and copies the 

of a new data read or write request. fiill "track*' of data to the cache memory 18 of the second 

If, however, at step 102, the second data storage system 16 50 ^^^^ storage system 16. The new data to be written is then 

determines that the data is not presently stored on the second written into the proper memory location in cache memory 18 

data storage system 16, the second data storage system 16 (the occurrence of the actual "write" command), the data 

wiU generate a request to the first data storage system 14 to table/map 24 updated (for example, to indicate that the data 

read the data, step 106. is in cache memory 18 [data in cache bit set], that a write is 

The command or request to read data from the first data 5S Pending on this data [write pending bit set], and that the data 

storage system 14 takes the same form as a read data elements have been migrated [data needs migration bits 

command which would be issued from the host 12. Thus, for re-set]) and the host or other central processing unit 12 

example, if the host 12 is an IBM or IBM compatible host informed that the write command is complete, 

or data processing device, the second data storage system 16 At some later time, the data in cache memory 18 which 

wiU issue an IBM compatible "read" command to the first 60 has been flagged as write pending is copied to a more 

data storage system 14. The channel and migrate processes permanent storage location, such as a disk drive, and the 

25,27 of the second data storage system 16 maintain a list of write pending bit reset. 

conamands native to the firet data storage system 14 and can TypicaUy, data write requests are perfonned to update 

easily convert command types, if necessary, from a first only a portion of the total or complete number of data 

command type issued by the host 12 and understood by the 65 elements stored in a predetermined data storage element or 

second data processing system 16, to a second command physical/logical confine (such as a disk drive track). The 

type understood by the first data storage system 14. present invention, however, also realizes that in some cases, 
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such as when the host or data processing unit 12 provides an second data storage system 16 issues a request to the first 
indication that both the data structure (format) as well as the data storage system 14 for the data element, step 206. Once 

actual data contents are to be updated, reading old data from received, the second data siui age system 16 stotcs the data 

the first data storage system 14 may be eliminated since all on the second data storage system 16 (typically in cache 

data and data format or structure will be updated with the 5 memory 18), step 208, updates the second data storage 

new write request. Such a data and format write command system data map/table 24, step 210, and returns to step 200 

is so infrequent, however, that the preferred embodiment determine whether or not there is a pending data read or 

contemplates that each write request will cause a write request from the host or other data processing system 
request to be read from the first data storage system 14, 

The method of present invention also aUows the second or 10 ^° embodiment, the present invention contemplates 

new data storage system 16 to provide transparent or "back- ^^^^ ""^y ^ desirable to "prefetch" data from the first data 

ground" data migration between the first data storage system ^^^^^^ second data storage system 16. For 

14 and the second data storage system 16 irrespective of or example, the migrate or copy process 27 may, using com- 

in parallel with ±e data transfer or migration caused by the "^^^^^ ^'^^ ^^^^ ^^^'^^^ ^y^^^"™ i^^^ ^ 

channel process which is serving the "channel" between the 15 prefetch or "sequential" data access request or command to 

host 12 and the second data storage system 16. Since the ^^^^ ^^""'^^^ ^y^^^°^ *° '^^^s^ ^^^^ storage 

goal of providing the second or new data storage system 16 ^y^*^"^ contmue to fetch or 'prefetch' a certain number 

is to generally provide enhanced or increased capabiUties to ^^^^ elements to the cache memory 18 of the second data 

the host or other data processing system 12, it is therefore ^y^^^^^ ^""^^ prefetching can significantly speed 

desirable to migrate the data as quickly yet as unobtmsively 20 the transfer of data between the first and second data 

as possible from the first to the second data storage system. ^^^^^^^ ^y^^^"^ ^^'^^ ^'y S'^^^^^y reducing the number of 

rp. -^u *i. u 1 J • * i_„ "read" commands which must be passed between the data 

Thus, with the background migrate or copy "task" or storage systems 

"process" 27, the method of the present invention which is , , ... . . . 

a series of software instructions executed by a central . ° a?oth«r embodmient, the migration process 27 may 

processing unit in the second data storage system 16 accord- " ^etermme that one or more read requests from the host 12 

ing to the present invention (such hardware and software as P*^' "{^ ^^"""^ °^ i!"'' 

is well known in the art. see for example the EMC Sym- '°f!^'^^' ^^l^ P'"'^^ ^7 may take the current 

metrix series 5500 data storage systems), the present method '^■"f requested by the host 12 and mcrease 

first determines whether the second data storage system 16 " a predetertnined number. For example the host 12 is 

is completely busy servicing read or write data requests from ^° cunenUy requiting daU from an address '411 ' the channel 

the host or other connected data processing system 12, step P*"?^ ^ ^5 wUl issue a read request to the first data storage 

200, FIG. 4. If the second data storage system 16 is "P'^'" ^* /"'.""l *''*'".1f 9^^^"'^y 

completely busy handling such requests to and from the host s"n"l'aneously, the channel process will pass an indication 

or data processing system 12 or completely busy handhng ■"'S'"!^ P''*??f,f ^^"^ prefetching or migrating 

other data input/output (I/O) operations in the second data t T ^'^'^"^''^^l " ^"^'^^ '"'S^'^ ^7 wUl 

storage system 16, fiirther processing does not take place but '° 'T* ? data storage system 16 gets 

instead the migrate process 27 awaits a "no busy" or ^^^^ °^ the channel process 25 and the actual data 

"available" indication from the operating system of the ^quests from the first data storage system 14 The channel 

second data storage system 16. process 25 will handle requests from the host 12 for data at 

„ , , , , . , 40 addresses 411 and 412. Subsequent requests will already be 

Once the second data storage system 16 not busy ^ ^.^^e in the second daU storage system 16 and quiddy 

handling mtemal mput/output (I/O) requeste or requests ^^^1^^ ^y the second data storage system 16. 

irom the host or data processing device 12, the second data ^ c - j .j. 

storage system 16 reads the data map/table 24. step 202 and ^"^^^^ description presents one embodiment of a 

determines which data elements have not been copied from "'"''"^ ^/f ''"'"^e system and method that allows a new or 

the fiist daU storage system 14 to the second dafa storage f^"*^ ''.f f"^" ^^''^^ '° '^'^^'^'^ '° ^ ^""^^ 

system 16 step 204 processmg device with essentially no time 

y lost in accessing the data stored on a first or donor data 

As previously mentioned, during initial configuration of storage system. During the process there is real time, on-line 

the second data storage system 16, before the second data availability of the data to the host or other connected data 

storage system comes "on Ime". the user or system engineer 50 processing device, so normal operations of a data center can 

will utilize a system configuration device 26, such as a proceed. Data not involved with transfers to or from a host 

personal computer or other input device, to configure at least migrates to the new data storage system concurrently with 

a porUon of the data storage locations 17fl-17n in the second on-line operations. In essence, FIGS. 1 through 4 depict one 

data storage system 16 to exactly emulate (i.e. have the same embodiment of a method and apparatus for connecting the 

memory addresses) the data storage system configuration of 55 new or replacement storage device to the host system with 

the first or older data storage system 14. Generally, the new jts existing or donor storage device to form a composite 

or second data storage system 16 wiU have a greater storage storage device. One transfer path enables transfers between 

capaaty than the first or "old" data storage system 14 and the host system and the composite storage device. A data 

therefore, additional storage areas or locations will become migration path migrates data from the existing storage 

avaUable- Therefore, if the first data storage system 14 device to the replacement storage device within the com- 

includes a predetermmed number of drives or volumes, each posite memory. A control in the replacement storage device 

dnve or volume havmg a certam number of tracks or controlstheoperationof the transfer path and data migration 

records, the second data storage system will be configured to path untH all data has migrated from the existing or donor 

imitate such a configuraUon. storage device to the replacement storage device. Thereafter 

Once the second data storage system 16 has determined 65 all host system transfer requests are processed in the replace- 
that at least one data element (such as a track) has not been ment storage device and the existing or donor storage device 
copied from the old or first data storage system 14, the can be removed from the system or assigned other functions. 
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ALTERNATE EMBODIMENTS 

^^ilc the foregoing cmbodimciit performs uU these 
functions, the process of configuring a system such as by 
disconnecting the old data storage system 14 in FIG. 1 firom 
the CPU/host 12 and connecting the CPU/host 12 to the new 
data storage system 16 takes some time, usually less than 
one hour, during which ail applications must suspend opera- 
tions. In some data centers such an interruption is not 
acceptable. In others, the preparation for such an interrup- 
tion can represent a formidable task. 

In some data centers the connections between a host and 
old data storage system may use only a portion of the 
available channels. For example, in some centers a host 
computer with four available channels might use only two 
channels for communicating with the old storage device 14. 
Such availability often exists when the old storage device 14 
connects to multiple host computers or when a single host 
computer connects to multiple storage devices. 

FIG. 5 depicts one such system that includes the host 
computer 12, an old, or donor, storage device 14, and a new, 
or target, storage device 16, converter 22 and system con- 
figuration device 26 of FIG. 1. In FIG. 5 it is assumed that 
the connection 19 represents one channel and that there are 
four available channel interfaces on each of the host com- 
puter 12 and donor storage device 14. FIG. 5 also depicts a 
second host computer 12 A with a connection 19 A as a 
second channel between the host computer 12A and the 
donor storage device 14. In this particular configuration each 
of the host computers 12 and 12A have two available 
channels and, assuming each of the storage units has four 
available channel connections, each storage unit has two 
unused channels. 

If such a configuration exists, then in accordance with 
another aspect of this invention, the composite storage 
device can be formed without any significant interruption of 
operations in the data center. The steps for performing such 
a non-disruptive transfer procedure begin with the connec- 
tion of the target storage device 16 to available channels on 
the donor storage device 14 in the same fashion as previ- 
ously indicated. However, the connection 19 remains intact 
while new connections 28 and 28A are established to the 
host computers 12 and 12A to unused paths or channels. 

Once these connections are completed and the system 
configuration device 26 has properly configured the target 
storage device 16, a non-disruptive transfer procedure 300 
shown in FIG. 6 begins. Step 301 verifies that all the 
appropriate steps have been completed to establish the 
appropriate configuration. If this is not done, step 301 
diverts to step 302 to complete that procediu-e. When the 
setup is complete, step 301 diverts to procedure 303 that 
performs a swapping task so subsequent I/O data transfer 
requests communicate through the paths 28 and 28 A of FIG. 
5 to the target storage device and do not communicate 
through the connections 19 and 19A. Step 304 in FIG. 6 
represents the initiation of any of the previously or subse- 
quently described data migration techniques. 

Essentially it becomes necessary to interrupt the operation 
of the host computer 12 and any additional host computer, 
such as the host computer 12A, connected to the donor and 
target storage devices in order to swap the I/O request from 
the donor storage device 14 to the target storage device 16. 
As known in the art, many data processing systems operate 
with an ability to run small batch programs with special 
instructions. One instruction for performing the swapping 
operation for an MVS system is: 
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S NDSDM, FROM-xxxx, TO-yyyy [SHARED-N/ 
WTOR/CKPT] 
liiis command establishes a procedure and can be started 
from a main frame operator's console for each host com- 

5 puter attached to the donor storage device. The NDSDM 
field is a mnemonic indicating that a data migration swap is 
being implemented. The "FROM" parameter identifies the 
donor storage device 14 in FIG. 5; the "TO" parameter 
identifies the target storage device 16. The "SHARED" 

10 parameter can have three values. "N" indicates that there are 
no shared host computers. This would be applied in a system 
for a data center as shown in FIG. 1. If two host computers 
connect to the donor storage device, as shown in FIG. 5, the 
operator initiates the transfer by selecting either a WTOR or 

15 CKPT parameter in an effort to assure that the switch of data 
transfer requests from the donor storage device 14 to the 
target storage device 16 occurs in a timely and coordinated 
fashion. 

Once the operator issues this command in step 305, in 
20 FIG. 7 as from the operator's console, step 306 determines 
whether the elements to be involved in the data migration are 
appropriate (i.e., have a valid configuration or syntax). Step 

307 determines whether the "FROM" and "TO" volumes are 
valid. As previously indicated each storage device can 

25 comprise one or more logical volumes. Typically a data 
migration will be made on a volume-by-volume basis. Step 

308 determines whether the "FROM" volume has any 
restrictions that preclude the data migration. If the volume is 
restricted, if the configuration is not valid or if either the 

30 "FROM" or "TO" volumes are not valid, step 309 terminates 
the task 303 and generates an appropriate error message. 

Assuming these tests are met satisfactorily, control trans- 
fers to step 310 in which the operator, each of the computers 
12 and 12A connected to the identified volumes suspends 

35 I/O operations as by using standard MVS services to issue 
an I/O ACTION STOP command. After this suspension 
occurs in step 310 the procedure 303 determines whether 
shared host computers are involved. If the data center has a 
configuration as shown in FIG. 1, step 311 determines that 

40 no sharing is involved and diverts control to step 312 that 
swaps the contents in the "FROM" and "TO" unit control 
blocks (UCB's). Then the system uses steps 313 and 314 to 
alter any duplicate volume identifications, typically by 
changing the identification of the volume in the donor 

45 storage device. This precludes multiple volumes with iden- 
tical identifications. Once this is accomplished, step 315 
reenables I/O operations from the host computer 12. All 
subsequent data transfer requests are handled by the target 
storage device 16 over the connection 28. 

50 If muhiple host computers are involved, step 312 requires 
a prior synchronization or coordination to assure that the 
swap of the "FROM" and "TO" UCB's occurs in all host 
computers connected to the logical volume at the same time. 
This precludes a situation in which different host computers 

55 operate with both the donor storage device 14 and the target 
storage device 16. In one approach it may be possible to use 
services within the host computers to effect the synchroni- 
zation. For example, check point services can synchronize 
events if an initialized control file on a separate device is 

60 shared by all the systems on-line to the FROM device. If this 
condition exists, the command issued in step 305 will 
incorporate the parameter SHARED =CKPT and operation 
will divert from step 311 to step 320 that initiates the 
operation. The procedure in FIG. 7 then awaits an indication 

65 of synchronization in step 321. If it is received within a 
predetermined time, control passes to step 312 to effect the 
switch over. If the synchronization does not occur within a 
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particular time, then step 321 diverts to step 322 that 
terminates the data migration procedure by reeaabling or 
resuming 1/0 operations with the donor storage device 14. 
Generally an enor message also appears at the operator's 
console. Step 322 also continues the processing of data 
transfer requests with the donor storage device 14. 

A second approach enables the transfer to occur manually 
from the system consoles. In this case SHARED=WTOR 
and steps 311 and 320 divert to step 323. If the SHARED 
parameter does not have the WTOR value, then, in the 
sequence shown in FIG. 7, a potential error exists because 
none of the accepted SHARED parameters has been 
received. Control diverts to step 322. 

When the WTOR value for the SHARED parameter is 
decoded, step 324 issues a WTOR (Write To OperatoR) 
command that establishes the necessary synchronization and 
returns a reply. Again this command must issue from all host 
computers that are sharing the donor storage device 14 or a 
volume in that device and target storage device 16. Each 
system responds to the receipt of the WTOR command by 
issuing an I/O ACTION STOP command and then by issuing 
a reply. When the replies from all the hosts indicate that 
everything is satisfactory, operations can continue. Step 325 
then diverts control to step 312. Otherwise step 325 diverts 
control to step 322. Thus the steps immediately after step 
310 determine whether SHARED host computers are 
involved in the data migration. If they are, the operations aire 
synchronized before the data migration begins. Once I/O 
operations are reenabled or resume, all the host computers 
involved with the storage devices thereafter direct all data 
transfer requests to the target storage device 16 and the 
donor storage device 14 is effectively removed from the 
network. 

As will be apparent, this procedxire can be completed 
within a matter of seconds and therefore does not interrupt 
the operation of the data center. Further it enables the 
transfer to occur without any of the data center preparation 
steps that have been necessary in order to effect an orderly 
transfer with other transfer techniques. 

Another alternative embodiment of this invention that can 
enhance operations minimizes the impact of any power 
failure or other problem that might disrupt the data migra- 
tion process. FIG. 8 depicts details of a system such as 
shown in FIG. 4 to particularly identify the new storage 
system with greater clarity. The new or target data storage 
unit 16 connects to a channel director 400 that in turn 
connects to one or more CPU/hosts 12 (not shown in FIG. 
8). A common bus 401 then connects the cache memory 18 
to disk directors. FIG. 8 depicts a first disk director 402 with 
disk drives 403; a second disk director 404 connects the bus 
401 to disk drives 405. Each disk director controls operation 
of its attached array of disk drives. Each disk drive may 
comprise one or more logical volumes, one such volume 406 
being shown as a component of one of the disks in the array 
403. 

As previously indicated the purpose of migrating data 
from the old data storage system 14 to the new data storage 
system 16 in FIG. 1 is generally to increase the size of the 
available storage. Consequently when it is desired to prevent 
any adverse response to a disruption during the data migra- 
tion phase, each volume 403 will have a size suflScient to 
accept the data from the old data storage system 14 plus a 
volume data map depicted as a set of tracks 407 in the 
volume 406, The volume data map 407 will contain infor- 
mation as shown in FIG. 3 limited to that information 
corresponding to the particular disk drive and logical vol- 
ume. Thus if the volume 406 corresponds to Volume 1 in 
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Device X as shown in FIG. 2, the map 407 would contain 
information concerning tracks 1 through N as depicted in 
FIG. 2, but no information concerning any other volume or 
disk unit. 

During data migration a data transfer request initiates an 
update data map/table procedure 116 in FIG. 3; the corre- 
sponding procedure 210 in FIG. 4 operates in the back- 
ground modes. FIG. 9 depicts a detailed version of those 
procedures. In step 410 the procedure updates the "need 
migration" flag 60 for the corresponding device, volume and 
track in the data map 24 as previously indicated. In accor- 
dance with this aspect of the invention, another step 411 
stages a write request to the volume data map 407. More 
specifically, if the NEED MIGRATION flag 60 in FIG. 2 
were changed for any of the tracks in volume 1 in FIG. 2, 
step 411 would stage a write request to alter the correspond- 
ing NEED MIGRAnON flag in the volume data map 407 of 
FIG. 8. 

Thus in accordance with this aspect of the invention two 
copies of the data map table are maintained. The first is the 
complete data map table 24 in FIG. 2 that is stored in the 
cache 18 of FIG. 1. The second data map table is distributed 
among the volumes of disk arrays such as disk arrays 403 
and 405. Consequently should any event occur, such as a 
power failure, that might cause the cache 18 to lose data or 
might corrupt data in the cache, it becomes a simple task to 
reconstruct the data map table 24 in the cache from the data 
that is permanently stored on the distributed volume data 
maps on the disks 403 and 405 and thereby continue the 
migration from the point at which the interruption occurred. 
This eliminates any need to migrate previously transferred 
valid data elements again. 

As another alternative embodiment it is possible to 
modify the channel process 25 and migrate process 27 in 
FIG. 1 so that their respective operations are controUed in 
response to certain statistical information that can be devel- 
oped during the migration process thereby to minimize 
response times to data transfer requests during the data 
migrations. In essence a copy subroutine runs in a back- 
ground mode to transfer data track by track in sequence from 
a starting location until all the data is migrated. This 
operation corresponds to the operation of the migrate pro- 
cess 27 in FIG. 1. If the host processor 12 issues a data 
transfer request (DTR), including either a read or write 
command, and the corresponding data is not located in the 
target storage device 16, a foreground mode is established 
that causes the copy subroutine to transfer the requested 
data. This operation in this mode corresponds to the opera- 
tion of the channel process 25 in FIG. 1. If a series of such 
data transfer requests establish a significant pattern of 
accesses to a localized area of the donor storage device 14, 
the parameters controlling the copy subroutine in the back- 
ground mode are altered to shift the background copying to 
the localized area in which the statistically significant pat- 
tern of requests occurred. 

FIG. 10 depicts apparatus in the form of registers that 
implement this invention shown in a memory block 200; as 
will be apparent, the registers may be located at different 
locations within the data storage system 16 as part of the 
migrate process 27. 

In the memory block 200 a STATISTICAL BLOCK SIZE 
register 201 records a number of consecutive blocks that will 
define a localized area. This is a fixed niunber that typically 
will be installed from the system configuration device 26. 

A STATISTICAL BLOCK CONTROL register 202 
includes an identification (ID) field 203 and a DTR NO field 
204. The ID field 203 contains the identification of the 
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Statistical block currently being evaluated; the DTR NO field 
204 acts as a counter that alters each time a data transfer 
request (DTR) is made to that statistical block. A STATIS- 
TICAL BLOCK TRANSFER MIN register 205, also set to 
an initial value by the system configuration device 26, 
defines a user-generated minimum number of consecutive 
data transfer requests needed to initiate a copy program 
transfer. That is, register 205 establishes a threshold value 
that defines the boundary between random accesses that 
cause no change in the operation during the background 
mode and repeated access that produce the background 
mode operating change. 

A COPY PROGRAM MIN BLOCK register 206 stores a 
minimum mimber of blocks, such as data tracks on a disk, 
thai should be moved before any relocation of the copy 
program can occur. Specifically, the number in this register 
establishes a dead band or minimum delay that must expire 
before the copy program can be moved in response to a 
series of DTR requests to another area. 

A COPY PROGRAM STARTING ADR register 207 
stores the starting address for the copy program. Typically 
this would be initialized to a first track. 

A COPY PROGRAM BLOCK ADR register 210 stores 
the current block address being transferred by the copy 
program. Typically this will be a track identification. In a 
sequential mode this register will be incremented or decre- 
mented to point to a successive address location after each 
transfer is complete. 

A COPY PROGRAM BLOCKS register 211 counts the 
number of blocks that have been Uansferred after the COPY 
PROGRAM STARTING ADR register 207 is updated or 
initialized. This controls the relocation of the program. It is 
set to the value stored in the COPY PROGRAM MIN 
BLOCK register 206. 

The remaining elements in the memory block 200 of FIG. 
10 include a copy subroutine 212, a background mode 
controller 213, a foreground mode controller 214 and an 
interruption flag 215. As will now be described, the con- 
trollers 213 and 214 establish and control the areas from 
which the copy subroutine in block 211 transfers data from 
the donor storage device 14 to the target storage device 16. 
The interruption flag 215 controls that transfer between 
modes. 

FIG. 11 depicts the various steps by which the background 
mode controller 213 and the copy subroutine 212 interact to 
transfer data on a track-by-track basis. Registers in the 
register set 200 are set to initial values in step 220. Then the 
program enters a loop comprising the remaining steps in 
no. 11 until all the NEED MIGRAHON flags 60 of HG. 
2 are set using step 221 as a loop control. As a first action 
in the loop, step 222 determines whether the STATISTICAL 
BLOCK INTERRUPTION flag 215 is set indicating that the 
copy subroutine 212 in FIG. 10 needs to be relocated. If that 
condition exists, control diverts to step 223 that updates the 
copy program parameters in registers 207 and 210 thereby to 
relocate the position of the copy subroutine to another track. 

If the STAnSTICAL BLOCK INTERRUPTION flag 206 
is not set or after the copy program parameters are updated 
in step 223, step 224 determines whether the NEED 
MIGRAnON flag 60 for the new track is set. If it is, step 
225 copies the track, or other block of data elements, from 
the donor or first data storage device 14 to the target or 
second storage device 16. In step 226 the system clears the 
NEED MIGRATION flag 60 for the corresponding track 
position. Steps 225 and 226 form the copy subroutine 212. 
When the NEED MIGRATION flag 60 for a track is not set, 
the block has l?een previously transferred so control diverts 
from step 224 directly to step 227. 
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Step 227 increments the value in the COPY PROGRAM 
BLOCK ADR register 210 and step 228 increments the 
COPY PROGRAM BLOCKS register 211. Thus, the back- 
ground mode controller 211 in FIG. 11 will, absent the 
setting of the STATISTICAL BLOCK INTERRUPTION 
flag 215, copy the tracks or data blocks fi-om the donor 
storage device 14 to the target storage device 16 in an 
ordered sequence. Moreover the transfers are non-redundant 
because once a data block is transferred to the target storage 
device 16, all further DTR commands for a data element in 
that block are handled exclusively by the target storage 
device 16. 

FIG. 12 depicts the operation of the foreground mode 
controller 214 that controls the response to a DTR (data 
transfer request) command, makes any necessary transfer 
and determines whether the accesses define a significant 
pattern that warrants setting the STATISTICAL BLOCK 
INTERRUPTION flag 215. As part of an initialization 
procedure 230 in FIG. 12, the system will initialize (1) the 
statistical block size, (2) statistical block control ID and 
DTR NO values, (3) the copy program minimum block size 
and (4) the copy program starting position in the correspond- 
ing registers in block 200 of FIG. 10. Step 231 waits for a 
host command. When a host command is received, step 232 
determines whether that command is a data transfer request 
(DTR) command. If not, step 232 branches to step 233 
where the command is processed. Thereafter the system 
awaits the receipt of a next command at step 231. 

Each time a DTR command is received, control branches 
from step 232 to step 234 to determine whether the target 
storage device 16 contains the requested data element. If it 
does, step 235 transfers the data element to the host com- 
puter in accordance with the DTR command. There is no 
requirement for any communication with the donor storage 
device 14. The response time then is the response time of the 
target storage device 16. 

If the requested data element is not in the target storage 
device 16, migration is necessary. Step 236 interrupts the 
operation of the background mode controller 213 in FIG. 11 
to transfer a track or other block containing the data element 
identified by the DTR command in step 237. In essence step 
237 calls the copy subroutine 212 in FIG. 10 and supplies the 
arguments or parameters necessary to effect the transfer. 

Next there is a determination of whether the access has 
established a significant pattern. In this particular 
embodiment, step 238 compares the statistical block iden- 
tification associated with the DTR command with the ID 
field 203 in the STARSTICAL BLOCK CONTROL register 
202. If the numbers are not the same, step 240 transfers 
control to step 241 that replaces the contents of the ID field 
203 with the corresponding statistical block identification 
for the DTR command. Control then returns to await the 
next host command at step 231. Thus the foreground con- 
troller 214 follows control path through step 241 in response 
to random DTR accesses. 

If the identification is the same as the identification in the 
field 203, step 240 branches to step 242. This branch 
represents an indication of localized access for this DTR 
command is to an area defined by the statistical block size 
in register 201 of FIG. 10. In step 242 the contents of the 
DTR NO field 204 are incremented. If the number in the 
field 204 is not above a threshold, step 243 diverts to loop 
back to await the next host command at step 231. If the 
number is above the threshold, indicating a significant 
pattern of accesses to a localized area, step 243 diverts to 
step 244 that compares the minimum copy block size in 
register 206 with the number of transfers that have occurred 
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as obtained from register 211. If the minimum block size has command will access data akeady in the target storage 

uoi been satisfied, step 245 diverts back to step 231 to wait device 16 so the response lime will be that of the target 

for the next host command. Thus no relocation of the copy storage device 16. 

subroutine 212 will occur until the minimum number of Graph 251 depicts an optimal data migration response 

transfers has been made from an existing localized area. 5 ctUA^e. It is assumed for this ctirve that it would be possible 

Once that minimum is reached, step 245 diverts to step 246 predict with certainty the locations accessed by the pattern 

that sets the interruption flag 215. Step 215 also generates commands. Relevant data is transferred initially so 

new copy program parameters and then restores the back- ^^^^ response time drops rapidly to the minimum value, 

ground mode of the copy procedure. ^^^^^ practice it is not always possible to make such 

When the interruption flag 215 is set and the background lo " P^^^iction. Graph 252 depicts a typically observed 

mode controUer 213 in FIG. 11 enhances a next iteration, |f'P°f Tl^^ T f 'fi' T 

* J * • *L * *L • * a ' . J been round that this invention significantly reduces the 

step 222 determines that the mteniiption flag 215 is set and ^^^^^ ^^^^^^ ^^^^^^ migration over the graph 

diverts control to step 223 lo update the copy subroutine 250, In many cases the actual response time graph approach 

parameters or arguments with the new copy program param- optimal graph 251. 

eters generated in step 246 of FIG. 12. This will relocate the is ConsequenUy this method and apparatus disclosed in 

copy subroutine to the statistical block corresponding to the piGS. 10 through 12 enables the efficient transfer of data 

localized area accessed by the sequential DTR commands. from one storage device to another in concert with other 

That is, the copy subroutine begins to transfer blocks or external operations as represented by DTR commands. The 

tracks sequentially from that initial operation at a first block transfers are particularly eflScient in a data migration sce- 

or track in the new statistical block or localized area that 20 nario where the data migration occurs in a transparent or 

exhibits the significant access pattern and continues transfers parallel mode and provides a minimal impact on response 

from that localized area until at least the minimum number times to DTR commands. 

of blocks have been transferred. The sequential transfer then Although the present invention is preferably implemented 

continues until the DTR commands estabhsh a statistically ^ software, this is not a limitation of the present invention 

significant pattern of accesses within another statistical 25 those well know in the art can appreciate that the present 

block. invention can be implemented in hardware of in various 

To summarize the operation of this invention, the copy combinations of hardware and software, without departing 

subroutine 212, essentially comprising steps 225 and 226 in the scope of the invention. Modifications and substi- 

FIG. 10, operates in response to calls from the background lutions by one of ordinary skill in the art are considered to 

mode controller 213 of FIG. 11 to move data on a track- 30 be within the scope of the present invention which is not to 

by-track, or other data block-by-data block basis, from the be limited except by the claims which follow, 

donor storage device 14 to the target storage device 16, If an This invention has been disclosed in terms of certain 

occasional or random access is requested by a DTR embodiments. It will be apparent that many modifications 

command, the foreground mode controller 214 in FIG. 12 can be made to the disclosed apparatus without departing 

interrupts the operation of the background mode controller 35 Ihe invention. Therefore, it is the intent of the appended 

213 in FIG, 11 to transfer the track or data block containing claims to cover all such variations and modifications as 

the requested data element to the target storage device 16. come within the true spirit and scope of this invention. 

Thereafter control passes back to continue the copy subrou- What is claimed as new and desired to be secured by 

tine calls from the background mode controller 213 accord- Letters Patent of the United States is: 

ing to the original sequence, 40 1- Data migration apparatus for migrating data from a 

If, however, successive DTR commands cause the fore- donor disk array storage system to a replacement disk array 

ground mode controller 214 to access data blocks concen- storage system while continuing to respond to write requests 

trated in a particular statistical block, the system predicts fro^i a host computer coupled to said replacement disk array 

that further requests will be made to that statistical block. storage system said apparatus comprising: 

The foreground mode controller 214 in FIG. 12 then alters 45 A) a data store, and 

the arguments used by the background mode controller 213 B) a migration control that connects to the host computer 

in FIG. 11 to shift the operation of the background mode and the donor disk array storage system thereby to 

controller 213 to the statistical block receiving the repeated redirect input-output requests from the host computer 

DTR requests. The minimum block size prevents another to the replacement disk array storage system and that 

shift of that operation until such time as a minimum number 50 migrates data from the donor disk array storage system 

of data blocks or tracks have been transferred. This process to the replacement disk array storage system under the 

continues then until all the NEED MIGRATION FLAGS 60 control of the replacement disk array storage system 

have been cleared indicating that all the data has migrated. and that responds to data write requests from the host 

When this occurs, step 221 in FIG. 11 transfers control to a computer to locations in the replacement disk array 

DONE procedure 247 that causes appropriate completion 55 storage system to which data has not yet been migrated 

messages to be generated. by transferring the data associated with the write 

FIG, 13 depicts, in graphical form, the comparison of host request to the replacement disk array storage system 

computer response time to DTR commands as a function of and by preventing the transfer of corresponding data 

data migration time. Graph 250 represents a typical response from the donor disk array storage system to the replace- 

scenario for random access requests. The graph indicates 60 ment disk array storage system, 

that initially there will be maximum response times and that 2. Data migration apparatus as recited in claim 1 wherein 

these response times will decrease to normal response times said migration control includes a copy program that transfers 

in a substantially linear fashion as the migration continues. the data from the donor disk array storage system to the 

The maximum response time represents the time required to replacement disk array storage system transparently to the 

complete a transfer from the donor storage device 14. 65 operations of the host computer. 

Essentially and intuitively, as more data migrates to the 3. Data migration apparatus as recited in claim 2 wherein 

target storage device 16 the more hkely it is that a DTR the replacement disk array storage system stores data in data 
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blocks of contiguous records and a write request transfers 
data for an entire block, said migration control including a 
write-responsive control that prevents the copy program 
from migrating the data in the corresponding data block 
from the donor disk array storage system to the replacement 5 
disk array storage system. 

4. Data migration apparatus as recited in claim 2 wherein 
the replacement disk array storage system stores data in data 
blocks of contiguous records and a write request transfers 
data for a portion of a data block, said migration control lo 
including a write-responsive control that causes said copy 
program to migrate the entire block of data from the donor 
disk array storage system to the replacement disk array 
storage system prior to said data transferring step and 
thereafter to prevent said copy program from transferring the is 
data associated with the write request to the portion of the 
entire block. 

5. A replacement disk array storage system for connection 
to a host computer to replace a donor disk array storage 
system containing data, said replacement disk array storage 20 
system comprising: 

A) a data store, and 

B) a migration control that connects to the host computer 
and to the donor disk array storage system thereby 
redirecting input-output requests to the replacement 
disk array storage system and that operates in a first 
mode by migrating data from the donor storage system 
to said data store concurrently with and independently 
of any input-output requests from the host computer, 
said migration control responding to a read request 
from the host computer during the data migration if the 
requested data has not yet been migrated to said data 
store by using a second, higher priority mode, to 
transfer the requested data from the donor disk array 
storage system and to provide the requested data to the 
host computer. 

6. A storage system as recited in claim 5 wherein said 
migration control migrates data from the donor storage 
system transparently to the operations of said host computer. 

7. A method of migrating data from a first disk array 
storage system, that responds to input-output requests 
including read requests from a host computer, to a second 
disk array storage system that is to replace the first disk array 
storage system while continuing to respond to read requests 
from the host computer, said method comprising the steps 
of: 

A) redirecting input-output requests from the host com- 
puter to the second disk array storage system whereby 
the first disk array storage system ceases to respond 
directly to input-output requests from the host com- 
puter; 

B) migrating data from the first disk array storage system 
to the second disk array storage system under the 
control of the second disk array storage system con- 
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currently with and independently of any input-output 
requests directed to the second disk array storage 
system by the host computer; 
C) responding to a data read request from the host 
computer to the second disk array storage system for 
data that has not yet been migrated to the second disk 
array storage system by transferring the requested data 
from the first disk array storage system through the 
second disk array storage system to the host computer 
whereby after the migration is complete the second disk 
array storage system replaces the first disk array storage 
system. 

8. A method recited in claim 7 wherein said data migrating 
step is transparent to the operations of the host computer. 

9. A method of migrating data from a donor disk array 
storage system, that responds to input -output requests 
including write requests from a host computer, to a replace- 
ment disk array storage system that is to replace the donor 
disk array storage system while continuing to respond to 
write requests from the host computer, said method com- 
prising the steps of: 

A) connecting said replacement disk array storage system 
to the host computer and to the donor disk array storage 
system to receive input-output requests from the host 
computer whereby the donor disk array storage system 
ceases to respond directly to input-output requests from 
the host computer, 

B) migrating data from the donor disk array storage 
system to the replacement disk array storage system 
under the control of the replacement disk array storage 
system concurrently with and independently of any 
input-output requests directed to the replacement disk 
array storage system; and 

C) responding to data write requests from the host com- 
puter to locations in the replacement disk array storage 
system lo which data has not yet been migrated by 
transferring the data associated with the write request to 
the replacement disk array storage system and by 
preventing the transfer of corresponding data from the 
donor disk array storage system to the replacement disk 
array storage system. 

10. A method recited in claim 9 wherein said data migrat- 
ing step is transparent to the operations of the host computer. 

11. A method as recited in claim 10 wherein the replace- 
ment disk array storage system stores data in data blocks of 
contiguous records and a write request transfers data for a 
portion of a data block, said responding step additionally 
including the step of migrating the entire block of data from 
the donor disk array storage system lo the replacement disk 
array storage system and thereafter transferring the data 
associated with the write request to the data block in the 
replacement disk array storage system. 
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